Image processing apparatus, method, and storage medium

ABSTRACT

A binary image of an input image is generated, and a character region within the binary image and a region surrounding each character are acquired as character segmentation rectangle information. A thinning process is executed on a region within the binary image which is identified based on the character segmentation rectangle information to acquire a thinned image. An edge detected image of the region identified based on the character segmentation rectangle information is acquired. Whether each character identified based on the character segmentation rectangle information is a character to be separated from a background by the binarization process or not is determined based on a result of a logical AND of the thinned image and the edge detected image.

BACKGROUND OF THE INVENTION

Field of the Invention

The present disclosure generally relates to image processing and, moreparticularly, to an image processing apparatus, an image processingmethod, and a storage medium storing a program for determining acharacter within an image.

Description of the Related Art

In recent years, an increased number of colored documents are madeavailable because of wide spreads of color printers and color scanners,for example. There are more chances to capture and store such documentsas electronic files by scanning and transmit them to a third party overthe Internet, for example. However, because direct storage of full-colordata may impose a large load on an apparatus and lines, such data mayneed to be compressed to reduce its data amount.

In the past, methods for compressing a color image may include, forexample, a method including converting a color image to a binary imagehaving a pseudo-gray scale by using an error diffusion method or thelike and compressing the binary image, a method including compressing inJPEG, and a method including converting a color image with 8-bit palettecolors for ZIP compression or LZW compression.

According to Japanese Patent Laid-Open No. 2002-077633, a characterregion contained in an input image is detected, and the detectedcharacter region is converted to a binary image, is MMR compressed(binary non-reversible compressed) and is stored in a file along withcharacter color information on characters therein. Furthermore, an imagehaving the character region filled with a surrounding color on the inputimage is JPEG compressed (non-reversible compressed) by reducing itsresolution and is stored in the file as a background image. The filecompressed by this compression method may provide the character regionin high quality and may contribute to a greater compression rate.

According to Japanese Patent Laid-Open No. 2002-077633, in order todetect a character region, whether each set of black pixels in a binaryimage acquired by binarizing an input image possibly corresponds to acharacter is determined based on the size (width or height) of the setof black pixels and whether sets of black pixels having an approximatelyequal size exist closely to each other.

On the other hand, application of such a method for performing theregion determination based on a binary image as disclosed in JapanesePatent Laid-Open No. 2002-077633 to an input image in which a characterand a background are difficult to be separated by simple binarizationmay result in difficult identification of pixels included in thecharacter. For example, when simple binarization is performed on a blackcharacter over a white background (character image having a largerdensity difference between the character and the background), thebackground pixels and the character pixels may be separated easily. Onthe other hand, when binarization is performed on a black character overa dark background (character image having a small density differencebetween a character and a background), the separation between thebackground pixels and the character pixels is difficult. Particularly,performing binarization on a character over a high-density backgroundwith a threshold value lower than the density of the background mayresult in a binary character image with characters degraded in black. Inthis case, when the size of the high-density background region isapproximately equal to the size of the character, the binary image inwhich the background and the character are degraded in black as a resultof the binarization may be wrongly determined as a character pixel part.For example, when a document in which a part of a character string ismarked with a thick marker pen is scanned and the scanned image isbinarized, the entire part marked with a marker pen may sometimes turnblack. When the size of the part marked with a marker pen is close tothe character size, the whole pixels of the part marked with the markerpen may have a state degraded in black as a result of binarization andmay thus be handled as one character. In other words, all black pixelsin a region degraded in black as a result of binarization may possiblybe handled as pixels of a character.

SUMMARY OF THE INVENTION

According to an aspect of the present disclosure, an image processingapparatus includes a binarizing unit configured to generate a binaryimage by executing a binarization process on an input image, a firstdetermining unit configured to determine a character region within thebinary image, a character segmenting unit configured to acquire a regionsurrounding each character contained in the character region ascharacter segmentation rectangle information, a thinning unit configuredto acquire a thinned image by executing a thinning process on a regionwithin the binary image, the region being identified based on thecharacter segmentation rectangle information, an edge detecting unitconfigured to acquire an edge detected image by executing an edgedetection process on the region identified based on the charactersegmentation rectangle information, a logical operation unit configuredto take a logical AND of the thinned image and the edge detected image,and a second determining unit configured to determine whether eachcharacter identified based on the character segmentation rectangleinformation is a character to be separated from a background by thebinarization process or not based on a result of the logical ANDperformed by the logical operation unit.

Further features of the present disclosure will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an image processing system.

FIG. 2 illustrates a hardware configuration of a multi-functionperipheral (MFP) according to a first exemplary embodiment.

FIG. 3 is a block diagram of a region determining unit according to thefirst exemplary embodiment.

FIG. 4 is a diagram for explaining a region determination according tothe first exemplary embodiment.

FIG. 5 is a block diagram of an image compressing processing unit.

FIG. 6 is a block diagram of an image decompression processing unit.

FIG. 7 illustrates input image samples and output image samples.

FIG. 8 is a flowchart for region determination according to the firstexemplary embodiment.

FIG. 9 illustrates an example of an input image according to the firstexemplary embodiment.

FIG. 10 illustrates an example of an input image according to a secondexemplary embodiment.

FIG. 11 is a block diagram of an edge detecting unit according to afourth exemplary embodiment.

FIG. 12 illustrates samples for edge extraction.

FIG. 13 is a flowchart of the edge extraction according to the fourthexemplary embodiment.

FIG. 14 is a first block diagram of an edge detecting unit according toa fifth exemplary embodiment.

FIG. 15 is a second block diagram of an edge detecting unit according tothe fifth exemplary embodiment.

FIG. 16 illustrates samples according to the first exemplary embodiment.

DESCRIPTION OF THE EMBODIMENTS

Various exemplary embodiments, features, and aspects of the disclosurewill be described in detail below with reference to the drawings.

First Exemplary Embodiment

FIG. 1 is a schematic view illustrating a system configuration accordingto a first exemplary embodiment. Referring to FIG. 1, a multifunctionperipheral (MFP) 101 and a computer (hereinafter, called a PC) 102 areconnected over a network 103.

Broken lines 104 and 105 represent processing flows. The broken line 104represents a processing flow to be performed by a user for reading apaper document by using a scanner of the MFP 101. In this case, the useris allowed to set a destination (such as the PC 102) to which a scannedimage is to be transmitted and define settings regarding the scanningand transmission by using a user interface (203 in FIG. 2) of the MFP101, which will be described below. As the settings, a user maydesignate a resolution, a compression rate, a data format (such as JPEG(Joint Photographic Experts Group), TIFF (Tag Image File Format), PDF(Portable Document Format), high compression PDF, and high compressionPDF (with an OCR (Optical Character Recognition) result)). In thisexemplary embodiment, it is assumed that high compression PDF (with anOCR result) is designated as a data format. The technical details ofhigh compression PDF will be described below. The broken line 105represents a processing flow for generating data by using a software orhardware function of the MFP 101 and transmitting the data to adesignated destination based on the designated settings. Here, becausean image having a file format such as PDF is transmitted to the PC 102,it may be viewed on a general viewer in the PC 102.

FIG. 2 illustrates a detail configuration of the MFP 101. The MFP 101includes a scanner unit 201 that is an image input device, a printerunit 202 that is an image output device, a control unit 204 configuredto generally control the MFP, and an operating unit 203 that is a userinterface. The control unit 204 is a controller connecting to thescanner unit 201, printer unit 202, and operating unit 203 andconnecting to a local area network (LAN) 209 on the other hand forinput/output of image information and device information. A centralprocessing unit (CPU) 205 is a processor configured to control theentire system. A read-only memory (RAM) 206 is a system work memoryusable by the CPU 205 for operations and also functions as an imagememory for temporarily storing image data. A random access memory (ROM)210 is a boot ROM and stores programs such as a system boot program. Astorage unit 211 is a non-volatile storage medium such as a hard diskdrive and stores system control software and image data. As used herein,the term “unit” generally refers to any combination of software,firmware, hardware or other component, such as circuitry, that is usedto effectuate a purpose.

An operating unit I/F 207 is an interface unit to the operating unit(UI) 203 and is used to output image data to be displayed on theoperating unit 203 to the operating unit 203. The operating unit I/F 207serves to inform the CPU 205 of information instructed by a user of theimage processing apparatus through the operating unit 203. A network I/F208 connects the image processing apparatus to the LAN 209 for datainput/output (for example, for transmitting a PDF compressed data toanother apparatus and receiving PDF compressed data from anotherapparatus). These units are provided on a system bus 216. An image businterface 212 connects the system bus 216 and an image bus 217 is a busbridge configured to transfer image data at a high speed and convert adata structure. The image bus 217 may be a peripheral componentinterconnect (PCI) bus, IEEE1394, or the like. The units described aboveare provided on the image bus 217. A raster Image processor (RIP) 213performs what-is-called rendering processing including analyzing a PDL(page description language) code and decompressing it to a bitmap imagehaving a designated resolution. A device interface (I/F) unit 214connects the scanner unit 201 being an image input device through asignal line 218 and connects the printer unit 202 being an image outputdevice through a signal line 219 and performssynchronization-based/asynchronization-based conversion on image data. Adata processing unit 215 generates PDF compressed data (515) byperforming high compression PDF and OCR processing. The generatedcompressed data (515) is transmitted to a destination (such as theclient PC 102) through the network I/F 208 and over the LAN 209. Thedata processing unit 215 is capable of decompressing compressed datareceived through the network I/F 208 and over the LAN 209. Thedecompressed image is transmitted to the printer unit 202 through thedevice I/F 214 and may be printed.

Description of Data Processing Unit 215

Next, a configuration of the image compressing processing unitimplemented by the data processing unit 215 in FIG. 2 and aconfiguration of the image decompression processing unit will bedescribed with reference to the block diagrams in FIGS. 5 and 6. Thedata processing unit 215 may be configured such that a processor mayexecute a computer program to cause the data processing unit 215 tofunction as corresponding one of the processing units in FIGS. 5 and 6,or a part or all of the processing units may be configured by hardwaresuch as an ASIC (Application-Specific Integrated Circuit) and anelectronic circuit.

A high compression PDF process, as described in Japanese PatentLaid-Open No. 2002-077633, determines the type of a given region foreach attribute and compresses the data by adaptively selecting binaryreversible compression with MMR (Modified Modified Read) andmulti-valued non-reversible compression with JPEG in accordance with theattribute of the region. In other words, MMR compression may beperformed on a character region, and JPEG compression may be performedon an image having its character region filled with a surrounding colorfor a greater compression rate and higher quality of the characterregion. The high compression PDF processing is an effective compressiontechnology for color or black-and-white multi-valued images. Accordingto this exemplary embodiment, whether a region which may be degraded asa result of binarization is a character region or not may be determined,the details of which will be described below. This allows determinationof an actual character region only as a subject of MMR compression.

FIG. 5 is a block diagram illustrating a configuration of the imagecompressing processing unit implemented by the data processing unit 215and illustrating processing units for compressing an input image togenerate a high compression PDF (with an OCR result).

A binarizing unit 502 generates a binary image from an input image 501that is a multi-valued image. In the binary image, a pixel having adensity greater than a threshold value in an input image may be a blackpixel, for example, while a pixel having an equal or lower density thanthe threshold value may be a white pixel, for example. (Apparently, sucha binarization result may not be represented in black and white but maybe represented in other colors or may be represented by 1, 0 or 0, 1without color). The binarizing unit 502 performs its processing for thepurpose of distinguishing a pixel having a density greater than athreshold value and a pixel having an equal or lower than the thresholdvalue. Other processes than binarization may be applied as far as thesame purpose may be achieved (such as ternarization andquaternarization). However, the following description assumes thatbinarization is performed by the binarizing unit 502. Binarization on aninput image 701 may result in a binary image 702. When the input imageis a polychrome multi-valued image, the binarization is only performedon a luminance (such as Y of YUV) of the multi-valued image.

A region determining unit 503 detects a character region and a pictureimage region from a binary image generated by the binarizing unit 502.Thus, character regions 704 and 706 and a picture image region 705 aredetected, for example. This processing is performed based on a publiclyknown region determination method (such as Japanese Patent Laid-Open No.06-068301). The processing may be summarized as follows, for example.

(1) An outline of 8-connected black pixels is traced on the binary image702 to extract a black pixel cluster (black cluster of pixels) that iscontinuously present in any direction of in 8 directions. The expression“8-connected” refers to continuous presence of pixels having a samecolor (black in this case) in one of eight directions of upper left,left, lower left, lower, lower right, right, upper right, and upperdirections. On the other hand, the expression “4-connected” refers tocontinuous presence of pixels having a same color in any of fourdirections of left, lower, right, and upper directions.

(2) If there is a black cluster of pixels larger than a predeterminedsize (such as a black cluster of pixels surrounding a region larger thana predetermined dimension) among the extracted black clusters of pixels,whether any white cluster of pixels exists within the region or not isdetermined. In other words, a white cluster of pixels is extracted bytracing an outline of 4-connected white pixels within the region. If theextracted white cluster of pixels is larger than a predetermined size,the outline of the black pixel is traced again similarly to extract ablack cluster of pixels. These steps are repeated until the cluster ofpixels has a size equal to or smaller than the predetermined size.

(3) The acquired black cluster of pixels is classified into a characteror a picture based on at least one of its size, shape, and black pixeldensity. For example, a black cluster of pixels having an aspect ratioclose to 1 (or fits into 1 plus or minus α where α is a fixed thresholdvalue such as 0.1) and having a size within a predetermined range (forexample, the number of pixels surrounded by the black cluster of pixelsis equal to or lower than 100 pixels) is determined as a black clusterof pixels including a character. The other black clusters of pixels aredetermined as clusters of pixels included in a picture.

(4) The distance between black clusters of pixels included in onecharacter is equal to or shorter than a predetermined distance (such as3 pixels), the black clusters of pixels are classified into one group. Abounding rectangle region including both of the black clusters of pixelsclassified into one group is determined as a character region (704,706). It should be noted that a black cluster of pixels included in acharacter with no other black cluster of pixels included in thecharacter exists within a predetermined distance, the black cluster ofpixels may form one group singly. Therefore, the bounding rectangleregion of the single black cluster of pixels is determined as acharacter region. The same processing as the processing described in (4)is performed on black clusters of pixels included in a picture.

(5) Locations of the regions and attribute identification information(character or picture) of the regions are output as determinationresults.

The processing in (1) to (5) outputs a determination result that theregions 704 and 706 are character regions and the region 705 is apicture image region. The region determining unit 503 has been describedup to this point.

A character segmenting unit 504 performs a segmentation process with acharacter segmentation rectangle on the character regions generated bythe region determining unit 503. FIG. 7 illustrates segmentation results710, 711, 712, and 713. The segmentation process includes the followingsteps.

(1) One of the character regions is selected (708 is selected, forexample).

(2) A projection in a horizontal direction is taken for one binary imageidentified based on the character region. More specifically, the numberof black pixels on a horizontally extending line is counted, and theresulting counted number corresponds to its projection. FIG. 7illustrates a taken projection 715. In the projection 715, a series oflines in the vertical direction having more black pixels than athreshold value is handled as one group. As the result, three groupsoccur. The three groups include a group of lines having ABCD, a group oflines having EFG, and a group of lines having H.

(3) A projection in the vertical direction is taken for the groups. FIG.7 illustrates the projection 716 taken for the lines having ABCD.

(4) In the projections for the groups, a series of lines in thehorizontal direction having more black pixels than a threshold value ishandled as one group. For example, in the projection 716, four groupsoccur. The four groups include a group of lines having A, a group oflines having B, a group of lines having C, a group of lines having D.

(5) The bounding rectangles of the groups of lines acquired in (4) aresegmented as character segmentation rectangles. As a result, forexample, a bounding rectangle of each character is segmented as acharacter segmentation rectangle. FIG. 7 illustrates results 711, 712,713, and 710 of the segmentation.

(6) The processing in (1) to (5) is repeated until no more unselectedcharacter region exists.

FIG. 7 illustrates images to be processed and image examples of resultsof the binarization, region determination, and character segmentationprocesses. FIG. 7 illustrates an image 701 that is an example of theinput image 501 and further illustrates a character image 7011 on awhite background, a character image 7012 on a background having a lowerdensity, and a character image 7013 on a higher density background. Inother words, the character images 7011 and 7012 are character imageshaving a greater density difference between their characters andbackgrounds, and the character image 7013 is a character image having asmaller density difference between its character and background.

FIG. 7 further illustrates an example binary image 702 of a result ofbinarization performed on the image 701 in the binarizing unit 502 andfurther exemplarily illustrates a character 7013 is degraded in black asa result of a binarization process with a lower threshold value than thedensity of the background.

In description of this exemplary embodiment, a character image degradedas a result of a binarization process (an image in which a backgroundand a character are difficult to separate even by performing abinarization process thereon because the density difference between thebackground and the character is small, such as a character density on abackground greater than a threshold value) will be called a “characterimage difficult to be separated from the background”. A character imagenot degraded as a result of a binarization process (an image in which abackground and a character are easy to separate by performing abinarization process thereon because the density difference between thebackground and the character is large, such as a black character on awhite or lower background than a threshold value) will be called a“character image difficult to be separated from the background”. Inother words, the “character image easy to be separated from thebackground” is an image of a character region having a character imagepart being black pixels and a background part, excluding the character,being white pixels as a result of a binarization process.

The image 703 is a result of a region determination process performed onthe binary image 702 in the region determining unit 503. As a result ofthe region determination process, the regions 704 and 706 are determinedas character regions, and the region 705 is determined as a pictureimage region. The character regions 707 and 708 are partial imagesextracted from the binary image 703 and determined as character regionsby the region determining unit 503. A schematically illustratedcharacter segmentation rectangle 709 is a result of a segmentationprocess performed by the character segmenting unit 504. A charactersegmentation rectangle 710 is segmented from inside of the characterregion 704. Character segmentation rectangles 711, 712, and 713 aresegmented from inside of the character region 706.

A region determining unit 2 (505) determines whether the character imagewithin the character segmentation rectangle segmented by the charactersegmenting unit 504 is a character degraded by a binarization process(character image difficult to be separated from the background) or not.Details of the determination to be performed by the region determiningunit 2 will be described below. Based on information on the characterregion determined as a “character image difficult to be separated fromthe background” by the region determining unit 2 (505), the characterregion information generated by the region determining unit 503 and thecharacter segmentation rectangle information generated by the charactersegmenting unit 504 are modified. In other words, information on thecharacter region determined as a “character image difficult to beseparated from the background” by the region determining unit 2 (505) isremoved from the character region information generated by the regiondetermining unit 503 and the character segmentation rectangleinformation generated by the character segmenting unit 504. This maysolve a problem that the character region determined as a “characterimage difficult to be separated from the background” is not determinedas a character and does not undergo MMR compression, which will bedescribed below, resulting in invisibility of the character image.

An MMR compression unit 506 extracts a binary image of the characterregion based on the character region information modified by the regiondetermining unit 2 (505) from the binary image generated by thebinarizing unit 502 (or extracts a binary image included in thecharacter segmentation rectangular region determined as a “characterimage easy to be separated from the background”). Then, the MMRcompression unit 506 performs MMR compression on the binary image of theextracted character region to generate a compression code 1 (511).

A reducing unit 507 performs a reduction process (resolution reductionprocess) on an input image 501 to generate a reduced multi-valued image(not illustrated).

A representative-color extracting unit 508 identifies a location ofpixels (black pixels) forming characters in the binary image based onthe character region information and character segmentation rectangleinformation modified by the region determining unit 2 (505). Arepresentative color of characters is calculated for each charactersegmentation rectangular region based on the identified location ofpixels of the characters and with reference to the color of thecorresponding location in the reduced multi-valued image so thatcharacter color information 513 of each of the characters is acquired.For example, a representative color may be an average or a weightedaverage of colors in a multi-valued image of pixels that have turned toblack in the binary image in the character segmentation rectangularregion or a most frequently occurring color among the pixels. Variousmethods for acquiring a representative color may be possible, but acolor in a multi-valued image of at least one pixel of pixels havingturned to black in a binary image in a character segmentationrectangular region is used for the calculation of a representativecolor.

A character-region filling unit 509 identifies a location of pixels(black pixels) included in characters in a binary image based on thecharacter region information and character segmentation rectangleinformation modified in the region determining unit 2 (505). Then, basedon the identified location of pixels, the pixels in the correspondinglocation in a reduced multi-valued image is filled with colorsurrounding it (hereinafter, called a surrounding color). Thesurrounding color may be an average value of pixel values of pixelssurrounding a character, and the pixel value of pixels is replaced bythe acquired surrounding color. Details of such a fill-up processperformed by the character-region filling unit are disclosed in JapanesePatent Laid-Open No. 06-068301.

A JPEG compression unit 510 performs JPEG compression on an image havingundergone the fill-up processing performed by the character-regionfilling unit 509 to generate a compression code 2 (514).

An OCR unit (516) performs publicly known character recognitionprocessing with reference to the character segmentation rectangleinformation generated in step 904 for the region determined as acharacter region by the region determining unit (503). A character code517 is a character code acquired by the character recognitionprocessing.

Here, while the MMR compression unit (506) performs MMR compression on aregion, determined as a character or a region determined as a “characterimage easy to be separated from the background” by the regiondetermining unit 2 (505), the OCR unit (516) performs OCR on a regiondetermined as a character region by the region determining unit (503).

Among such regions, because a “character image easy to be separated fromthe background” corresponds to a partial region of a region determinedas a character region by the region determining unit (503), the regionof a “character image easy to be separated from the background” isnarrower. In other words, a target region subject to OCR is wider whilea target region subject to MMR compression is narrower.

A reason why a target region subject to OCR is wider is that even when atarget region subject to OCR contains a region that does not actuallycorrespond to a character, an unnecessary character code may only beacquired, which is not a significant problem (or such a unnecessarycharacter code may be deleted if it is considered so). On the otherhand, performing MMR compression on a region that does not actuallycorrespond to a character may cause deterioration in image quality inthe region. For that reason, a wider region is subject to OCR while anarrower region is subject to MMR compression.

Thus, compressed data a file in PDF format (515) is generated whichcontains the compression code 1 (511) acquired from a component,modified character region information (512), character color information(513), compression code 2 (514), and character code (517). The generatedfile in PDF format is transmitted to a destination designated by a user,as described above.

FIG. 6 is a block diagram illustrating a configuration of an imagedecompression processing unit configured to decompress PDF compresseddata transmitted from another apparatus. The processing illustrated inFIG. 6 may be executed for decompressing and printing a compressed data,for example. Here, an example will be described in which compressed datatransmitted from another apparatus is identical to the compressed datafile 515.

An MMR decompression unit 601 performs MMR decompression processing onthe compression code 1 (511) contained in the compressed data (515) fileto reproduce a binary image. A JPEG decompression unit 603 performs JPEGdecompression processing on the compression code 2 (514) to reproduce areduced multi-valued image. An enlarging unit 604 performs enlargingprocessing on the reduced multi-valued image decompressed by the JPEGdecompression unit (603) to generate a multi-valued image having anequal size to the size of the input image 501 before it is compressed.

A synthesizing unit 602 allocates, with reference to the characterregion information (512), a color corresponding to character colorinformation (hereinafter, called a character color) (513) to blackpixels in the binary image decompressed by the MMR decompression unit.The binary image to which the character color is allocated issynthesized over the multi-valued image generated by the enlarging unit604 to generate a decompressed image 605. For this synthesis, atransparent color is allocated to white pixels in the binary image sothat the background multi-valued image may be transparent. In thismanner, the image decompression processing unit decompresses compresseddata generated by the image compressing processing unit to generate thedecompressed image 605. The decompressed image 605 is transmitted to andis printed by the printer unit 202 through the device I/F 214. It shouldbe noted that the image decompression processing unit ignores thecharacter code 517. This is because a character code is not necessaryfor printing decompressed image. A character code is necessary for notthe MFP 101 but an apparatus such as the client PC 102 which displaysthe decompressed image 605 on a display device. Therefore, the MFP 101ignores the character code 517. More accurately speaking, a charactercode is necessary for a user who is using the PC 102 rather than the PC102 itself. A character code is used for cutting and paste and editing acharacter string.

Next, details of processing to be executed by the region determiningunit 2 (505) will be described. The region determining unit 2 (505)determines whether a character image within a character segmentationrectangle is degraded by binarization based on an binary image generatedby the binarizing unit 502, a reduced multi-valued image generated bythe reducing unit 507 and character segmentation rectangle informationgenerated by the character segmenting unit 504. It should be noted thatthe input image 501 may be used instead on the reduced multi-valuedimage if the input image 501 is held in the storage unit 211.

A detail configuration of the region determining unit 2 (505) will bedescribed with reference to FIG. 3. For easy understanding of thefollowing description, character image examples in FIG. 4 will also bereferred. FIG. 4 illustrates a character image 401 having a character ona white background (with a large density difference between thebackground and the character). When the binarizing unit 502 binarizesthe image 401, a binary image 402 is acquired. A character image 406 hasa character on a dark background (with a small density differencebetween the background and the character). When the image 406 isbinarized with a lower threshold value than the density of thebackground 407, a black degraded image 408 is acquired. Because asimilar image to the image 402 is acquired by binarizing a character ona background having a light density with a threshold value between thedensity of the background and the density of the character, thedescription will be omitted. Images 403 to 405 and 409 to 411 will bedescribed below.

The region determining unit 2 (505) includes a thinning unit 301, anedge detecting unit 302, a logical operation unit 303, an edge countingunit 304, and a number-of-edges comparing unit 305.

The region determining unit 2 (505) extracts internal edge pixels of aregion having a density greater than a threshold value (or black regionsin images 402 and 408) (1). If the number of extracted edge pixels islower than a threshold value, it is determined as a “character imageeasy to be separated from the background” (2). If the number ofextracted edge pixels is equal to or greater than the threshold value,it is determined as a “character image difficult to be separated fromthe background” (2).

For example, no edge pixels exist within the black region in the image402. On the other hand, there are edge pixels (edge pixels of thecharacter “H” in the image 410) within the black region in the image408. The term “edge pixel” refers to an edge pixel extracted from amulti-valued image (input image) rather than an edge pixel extractedfrom a binary image.

One configuration will be described below for implementing the steps (1)and (2) though an embodiment of the present disclosure is not limited tothe configuration. Other possible configurations will also be describedbelow.

The thinning unit 301 executes thinning process on a binary image incharacter segmentation rectangles with reference to charactersegmentation rectangle information. The thinning process thins a blackcluster of pixels by trimming outer 2 pixels of a black cluster ofpixels within a binary image (or replacing a black pixel on an outlineof a black cluster of pixels by a white pixel). For example, pixelswithin a binary image contained in a one target character segmentationrectangle may sequentially be handled as a pixel of interest to bescanned by utilize a 5×5 window. If at least one white pixel existswithin the 5×5 window, the pixel of interest (at the center of the 5×5window) may be replaced by a white pixel for implementing the thinningprocess. Here, performing the thinning process on the binary image 402may result in a thinned image 403, for example. Performing the thinningprocess on the binary image 408 may result in a thinned image 409, forexample.

The edge detecting unit 302 performs edge detection on an input reducedmulti-valued image in character segmentation rectangles with referenceto character segmentation rectangle information. An image in which apixel determined as an edge is represented by a black pixel and a pixelnot determined as an edge is represented by a white pixel will be calledan edge detected image. Because the edge detection may be based on apublicly known method, the following processing may be possible thoughdescription on details will be omitted. For example, differentialfiltering is executed on a luminance component of a reduced multi-valuedimage to acquire edge intensities of pixels. A pixel having an edgeintensity equal to or greater than a predetermined threshold value isrepresented by a black pixel, and a pixel having an edge intensity lowerthan the predetermined threshold value is represented by a white pixelso that an edge detected image may be generated. However, an edgedetection method according to a fourth exemplary embodiment, which willbe described below, may be used for achieving edge detection with highprecision.

Performing an edge detection process on a reduced multi-valued image,not illustrated, acquired by reducing the input image 401 may result inan edge detected image 404. Performing the edge detection process on areduced multi-valued image, not illustrated, acquired by reducing theinput image 406 may result in an edge detected image 410. If a reducedmulti-valued image, not illustrated, acquired by reducing the inputimage 401 or 406 has ½ of the resolutions of an input image, the images404 or 410 may have ½ of resolutions of the corresponding input image.However, those images are illustrated as having equal sizes forsimplification. When the storage unit 211 holds the input image 401 or406, the edge detection may be performed on the input image 401 or 406instead on a reduced multi-valued image.

The logical operation unit 303 is configured to take a logical AND of athinned image generated by the thinning unit 301 and an edge detectedimage generated by the edge detecting unit 302 to generate a logical ANDimage. More specifically, only when a thinned image generated by thethinning unit 301 contains a black pixel and a black pixel exists at thecorresponding location in the edge detected image generated by the edgedetecting unit 302, taking a logical AND of them results in a blackpixel. When an edge detected image generated by the edge detecting unit302 has ½ of resolutions of the thinned image, 0-order interpolation maybe performed on the edge detected image to have equal resolutions asthose of the thinned image before a logical AND is taken. Alternatively,a thinned image may be thinned out to have equal resolutions to those ofthe edge detected image before a logical AND is taken. Taking a logicalAND of the thinned image 403 and the edge detected image 404, a blackpixel does not exist within a logical AND image 405 fundamentally(though some black pixels may remain due to an influence of noise)because a black pixel in the thinned image 403 and a black pixel in theedge detected image 404 do not exist at an identical location. On theother hand, taking a logical AND of the thinned image 409 and the edgedetected image 410, a black pixel remains in a part corresponding to acharacter as in a logical AND image 411. From this, there is acharacteristic that a fewer black pixels exist within a logical ANDimage for a character image easy to be separated from the background”while more black pixels exist within a logical AND image for a“character image difficult to be separated from the background”.

An image 412 is acquired by overlaying the thinned image 403 and theedge detected image 404 with each other. The image 412 has black pixels413 corresponding to the thinned image 403 and has black pixels 414corresponding to the edge detected image 404. Because the black pixelsin the thinned image 413 and the black pixels in the edge detected image414 do not exist at an identical location, taking a logical AND of themdoes not generate black pixels.

The edge counting unit 304 is configured to count, as a number of edges,the number of black pixels in a result (logical AND image) of a logicalAND taken by the logical operation unit 303.

The number-of-edges comparing unit 305 is configured to compare thenumber of edges in a given image counted by the edge counting unit 304and a predetermined threshold value to determine whether the image is a“character image easy to be separated from the background” or a“character image difficult to be separated from the background”. Inother words, if the counted number of edges is lower than thepredetermined threshold value, the image is determined as a “characterimage easy to be separated from the background (character image notdegraded by binarization)”. If the counted number of edges is equal toor greater than the predetermined threshold value, the image isdetermined as a “character image difficult to be separated from thebackground (character image degraded by binarization)”.

If the pixel width of a black cluster of pixels is smaller than athinning width of a thinning process, performing the thinning processmay result in no black cluster of pixels within the binary image. Forexample, when a black cluster of pixels in a binary image is a thin linecharacter having a 3-pixel width and a thinning process with a 4-pixelthinning width is performed thereon, no black cluster of pixels remains.In a case where no black cluster of pixels remains as a result, theprocessing to be performed by the edge detecting unit 302, logicaloperation unit 303, edge counting unit 304 and number-of-edges comparingunit 305 may be omitted from viewpoint of the improvement of processingspeed. This is because counting the number of edges in a logical ANDimage with a thinned image apparently results in 0 even when if edgedetecting unit 302 detects edge pixels. Because the counting result 0means that the number of edges is lower than the predetermined thresholdvalue, the image may be determined as a “character image easy to beseparated from the background (character image not degraded bybinarization)”.

Therefore, in a case where all of black pixels in a subject charactersegmentation rectangle disappear due to a thinning process performedthereon, the image in the character segmentation rectangle is determinedas a “character image easy to be separated from the background(character image not degraded by binarization)” without performing theprocessing in the edge detecting unit 302 to the number-of-edgescomparing unit 305. When the processing to be performed by the units 302to 305 is omitted, the processing in the thinning unit 301 to thenumber-of-edges comparing unit 305 is performed on a next charactersegmentation rectangular region. The reason for omitting the processingmay be explained as follows. That is, such disappearance of black pixelsdue to a thinning process performed thereon may indicate that width ofsuch black pixels is thin and the black pixels generally corresponds toa character or a line though a width of black pixels in a binary imageis significantly narrow. Therefore, it may be explained that a targetcharacter segmentation rectangular region may be determined as a“character image easy to be separated from the background (characterimage not degraded by binarization)” without performing the processingfrom viewpoint of processing speed.

Alternatively, in a case where all black pixels within a binary imagedisappear, the number of pixels to be trimmed may be reduced. Forexample, replacing a pixel of interest (center of a 5×5 window) by awhite pixel because one white pixel exists within a 5×5 window mayresult in replacement of all black clusters of pixels by white pixels.To avoid this, the window size may be reduced for processing with a 3×3window.

Having described that the number of edges counted by the edge countingunit 304 is compared with a predetermined threshold value, a valueacquired by dividing the number of edges by the number of black pixelsin a thinned image may be compared with a predetermined threshold value.This may allow appropriate determination independent of the size of thecharacter segmentation rectangular region. The number of edges may bedivided by number of all pixels included in the character segmentationrectangular region or the number of black pixels after the rectangularregion is binarized. However, for the highest precision, the number ofedges may be divided by the number of black pixels in a thinned image.From this, the proportion of edges present in an internal part of thebinary image (internal part within a dark region) may be learned. Agreater proportion thereof means that there is a high proportion ofedges in the inner part of the binary image and therefore that there isa high possibility that the binary image is not a character.

Next, steps to be executed by the data processing unit 215 will bedescribed with reference to the flowchart in FIG. 8. FIGS. 2, 3, and 5are further referred in the following description. The regiondetermining unit 2 (505) executes steps 905 to 911 in FIG. 8.

In step 901, the binarizing unit 502 executes the binarization processon the input image 501.

In step 902, the region determining unit 503 executes the regiondetermination process on the binary image and identifies regionscontained in the binary image and determines whether each of theidentified regions is a character region or non-character region.

In step 903, the regions having undergone region determination by theregion determining unit 503 are handled sequentially one by one as aregion of interest. If the region of interest is a region determined asa character region by the determining unit, the processing moves to step904. If it is a region determined as a non-character region, theprocessing moves to step 913.

In step 904, the character segmenting unit 504 performs charactersegmentation on an image within the region of interest to generatecharacter segmentation rectangle information.

In step 916, the OCR unit 516 performs a publicly known characterrecognition process on a region determined as a character region by theregion determining unit (503) referring to the character segmentationrectangle information generated in step 904.

In step 905, the thinning unit 301 executes a thinning process on eachbinary image resulting from the binarization in step 902 within thecharacter segmentation rectangle referring to the character segmentationrectangle information generated in step 904.

In step 906, the edge detecting unit 302 executes an edge detectionprocess on each reduced multi-valued image within the charactersegmentation rectangle (or an input image within the charactersegmentation rectangle) by using the reduced multi-valued imageresulting from the reduction of an input image (or the input image 501)and character segmentation rectangle information generated in step 904.

In step 907, the logical operation unit 303 takes a logical AND of thethinned image generated by the thinning unit 301 in step 905 and theedge image generated in step 906.

In step 908, the edge counting unit 304 counts black pixels in an imageresulting from the logical AND (AND) performed by the logical operationunit 303 in step 907 to acquire the number of edges. Here, the acquirednumber of edges may be divided by an area of the character segmentationrectangular region (the total number of pixels within the charactersegmentation rectangular region) to be normalized for acquiring thenumber of edges per a unit area. This advantageously allows comparisonwith a threshold value in step 909 independently from the size of thecharacter segmentation rectangular region.

Next, in step 909, the number-of-edges comparing unit 305 compares thenumber of edges counted in step 908 and a threshold value th. Here, ifthe number of edges is greater than the threshold value th, thecharacter segmentation rectangular region of interest is determined as a“character image difficult to be separated from the background” in step910. If the number of edges is equal to or lower than the thresholdvalue th, the character segmentation rectangular region of interest isdetermined as a “character image easy to be separated from thebackground” in step 911.

In step 912, the character segmenting unit 504 determines whether allcharacter segmentation rectangles within the character region ofinterest have been processed. If so, the processing moves to step 913.On the other hand, if there still remains an unprocessed charactersegmentation rectangle, the next character segmentation rectangle is setas a next rectangle of interest in step 914, and the processing returnsto step 905.

If it is determined in step 913 that the determination on all of theregions has ended, the processing ends. If it is determined that therestill remains an unprocessed region, the next unprocessed region is setas a region of interest in step 915, and the processing returns to step903.

In this manner, the region determining unit 2 (505) determines with highprecision whether each of character segmentation rectangular regions isa “character image easy to be separated from the background” or a“character image difficult to be separated from the background” based onthe number of black pixels (the number of edges) resulting from thelogical AND of its thinned image and its edge detected image.

Because a “character image difficult to be separated from thebackground” (713 in FIG. 7, for example) is removed from the characterregion information, it is not processed by the MMR compression unit 506.In other words, a “character image difficult to be separated from thebackground” is compressed by the JPEG compression unit 510 along withits background image without binarization.

In this manner, whether a given image is a character image degraded bybinarization or not is determined. Thus, degradation of the characterimage may be prevented.

Having described that according to this exemplary embodiment onecharacter “H” as in images 406 and 408 in FIG. 4 is determined as a“character image difficult to be separated from the background(character image degraded by binarization)”, for example, the embodimentis not limited thereto. For example, two or more characters may behandled as in an input image 1001 in FIG. 9. Binarization of the inputimage 1001 results in a binary image 1002. A character image degraded bybinarization is not necessarily rectangular but may be an image having apart of the character image degraded as in an image 1003 in FIG. 9, forexample. Binarization on the input image 1003 results in a binary image1004.

Next, another configuration of the region determining unit 2 (505) willbe described.

According to another configuration, (A) an image input to the regiondetermining unit 2 is first divided into a region having a greaterdensity than a threshold value and a region having an equal or lowerdensity than the threshold value (by using binarization, ternarizationor other method). As a result, regions 402 and 408 may be acquired, forexample.

(B) Edge pixels are extracted (by using an extraction method asdescribed above) from a region determined as having a density greaterthan the threshold value (H region in the image 401 or an entire regionof the image 406) in the input image. This edge extraction process isnot performed on an end part (such as one or two pixels inside from theend) of a region determined as having a density greater than thethreshold value. In other words, according to the configuration in (B),edge pixels at a predetermined distance or longer (inside) from an endpart of a region determined as having a density greater than thethreshold value are extracted. According to an alternativeconfiguration, edge pixels including in such an end part (pixels not ata predetermined distance or longer) may be extracted, and then edgepixels in such an end part are removed. Thus, the same result as theresulting images 405 and 411 may be acquired. While the predetermineddistance in this example is 3 pixels, it may be other values.

(C) Subsequently, the number of the resulting edge pixels is counted,and whether the number of the edge pixels is greater than the thresholdvalue th or equal to lower than the threshold value th is determined.

Thus, the same result (or a result of determination whether “characterimage difficult to be separated from the background” or “character imageeasy to be separated from the background”) as that of the aforementionedmethod may be acquired. Edge pixels may be extracted from an entireimage input to the region determining unit 2, instead of the processingin (B). In this case, an end part of a region determined as having adensity greater than the threshold value and a region having an equal toor lower than the threshold value are excluded from edge pixelsextracted from an entire input image. Thus, the same result as that ofthe configuration (B) may be acquired.

The processing in characters on a result of the character segmentationin step 904 performed by the character segmenting unit 504 has beendescribed according to this exemplary embodiment. The processing may beperformed on sectional regions acquired by segmenting each of charactersinstead of the processing in characters. For example, a region may beequally divided into four sectional regions for the character segmentingunit 504, and the processing may be performed on each of the regions.For example, FIG. 16 illustrates images 1300 to 1304 which are acquiredby equally segmenting the character segmented region 406 into 4. Theprocessing is performed on each of the images 1300 to 1304.Alternatively, the determination may be performed on a center part of acharacter segmented region (such as only 60% of a center part of acharacter-segmented region). For example, FIG. 16 further illustrates animage 1305 which is an extraction of 60% of a center part of thecharacter-segmented region 406, and the processing may be performed onthe image 1305. A combination of the determination oncharacter-segmented regions and the determination on thecharacter-segmented regions and/or a center part thereof may allowdetermination of whether the image is a “character image easy to beseparated from the background” or a “character image difficult to beseparated from the background”.

Second Exemplary Embodiment

According to the first exemplary embodiment, a region determined as a“character image difficult to be separated from the background” by theregion determining unit 2 (505) is not subject to the MMR compressionprocess. According to a second exemplary embodiment, the regiondetermining unit 2 (505) re-executes a binarization process with highprecision based on a different algorithm from that of the binarizingunit 502 on a region determined as a “character image difficult to beseparated from the background”, and pixels of a character image part maybe separated from a background. In this case, performing the MMRcompression process on a character region resulting from there-binarization process with high precision may contribute toimprovement of image quality of the character region. For example,because the region 713 in FIG. 7 is determined as a “character imagedifficult to be separated from the background”, a region 7013 in theinput image 701 corresponding to the region 713 is only binarized with adifferent threshold value from that for other regions. As a result, abinary image like the image 714 in FIG. 7 may be generated, and thecharacter region may be MMR compressed. An example of there-binarization process with high precision may be a method forperforming a binarization process by using an average value of thedensity or luminance of a subject region as the threshold value insteadof a binarization process by using a fixed threshold value.

Third Exemplary Embodiment

According to the first exemplary embodiment, an input image having arelatively higher character quality like the image 401 in FIG. 4 hasbeen described, for example. However, performing the edge detectionprocess on an image (such as a scan document or a compressed image)having a poor character quality and more noise like the image 1101 inFIG. 10 may cause many edges within the character like an image edgedetected image 1102 in FIG. 12. Such edge appearance may become moresignificant particularly within a large character.

Here, edges within a character may easily remain in a logical AND image1104 acquired from the edge detected image 1102 and a thinned image1103. Many edges remaining inside of a character may read wrongdetermination of an actual “character image easy to be separated fromthe background” as a “character image difficult to be separated from thebackground”.

According to a third exemplary embodiment, increasing the amount ofreduction of the thinning process performed by the thinning unit 301 mayreduce edges remaining inside a character in a case where a subjectcharacter-segmented region has a larger size. The processing will bedescribed with reference to images 1105 to 1112 in FIG. 10.

An input image 1105 having poor character quality and more noisecontains a small character. An edge detected image 1106 is a result ofan edge detection process executed on the image 1105 having a smallcharacter. A thinned image 1107 is a result of a thinning processexecuted on the image 1105 having a small character. The thinningprocess may use a 5×5 window and include replacing a pixel of interest(center of a 5×5 window) by a white pixel if the 5×5 window has at leastone white pixel.

A logical AND image 1108 is a result of a logical AND of the edgedetected image 1106 and the thinned image 1107. Here, even a smallcharacter in an input image having poor character quality and more noisehas fewer edges, compared with the logical AND image 1104 having a largecharacter.

A large character image 1109 (identical to the character image 1101) isin an input image having poor character quality and more noise. An edgedetected image 1110 is a result of the edge detection process executedon the large character image 1109. A thinned image 1111 is a result ofthe thinning process executed on the large character image 1109. Thethinning process on a large character image may use a 9×9 window andinclude replacing a pixel of interest (center of a 9×9 window) by awhite pixel if the 9×9 window has at least one white pixel. In otherwords, changing the size of a window based on the size of a subjectcharacter image (size of a subject character-segmented region) mayincrease the amount of reduction of the thinning process. The size ofthe window is given for illustration purpose only and is not limited to5×5 or 9×9.

A logical AND image 1112 is a result of a logical AND of the edgedetected image 1110 and the thinned image 1111. The logical AND image1112 has fewer edges than the logical AND image 1104. Therefore, even animage having a large character with much noise may be determined as a“character image easy to be separated from the background” by reducingthe amount of reduction of the thinning process if the size of thecharacter image is large.

According to the third exemplary embodiment, the control of the amountof reduction of the thinning process to be performed by the thinningunit based on the size of a subject character-segmented region mayreduce an influence of noise, for example, and allow the determinationwith high precision even in a case where an input image is a scandocument, as described above.

Fourth Exemplary Embodiment

Next, details of the processing to be performed by the edge detectingunit (302) within the region determining unit 2 (505) in FIG. 5 will bedescribed with reference to FIG. 11. The edge detecting unit (302)includes a variance value detecting unit 1001, an edge determinationthreshold value calculating unit 1002, and an edge extracting unit 1003.The processing to be performed by the edge detecting unit (302) will bedescribed in more detail further with reference to FIG. 12. FIG. 12illustrates input images 1101, 1102, and 1103 segmented in charactersegmentation rectangles with reference to the character segmentationrectangle information, like the images 401 and 406 illustrated in FIG.4. The input images 1101, 1102, and 1103 are image examples havingdifferent signal values from each other when they are acquired by thescanner unit 201. More specifically, they have signal values representedby a L*a*b* color system where L* represents brightness, and a* and b*represent chromaticity. Notably, while this embodiment applies a L*a*b*color system, an embodiment of the present disclosure is not limitedthereto. For example, the same processing may be performed with a signalvalue in a different color space such as an RGB (red, green, blue) colorsystem. The region 1104 of the image 1101 has a signal value of {L*, a*,b*}={1128, −50, +30}. The region 1105 has a signal value of {L*, a*,b*}={128, +50, −60}.

FIG. 12 illustrates an example in which there is a difference in largesignal value between the regions 1104 and 1105. On the other hand, theregion 1106 in the image 1102 has a signal value of {L*, a*, b*}={128,−50, +30}. The region 1107 has a signal value of {L*, a*, b*}={128, −60,+30}. FIG. 12 illustrates an example in which there is a smalldifference in signal value between the regions 1106 and 1107. The region1108 in the image 1103 has a signal value of {L*, a*, b*}={128, −50,+30}. The region 1109 has a signal value of {L*, a*, b*}={128, −52,+30}. FIG. 12 illustrates an example in which there is substantially nodifference in signal value between the regions 1108 and 1109. Forexample, in a case where the edge detecting unit (302) performs an edgedetection based on a result of comparison in signal value simply betweenadjacent pixels or an edge detection by filtering, instead of theaforementioned configuration may cause the following problems. That is,with some threshold values, an edge may be acquire at a boundary betweenthe regions 1104 and 1105 in the image 1101 while no edge may beacquired at a boundary between the regions 1106 and 1107 in the image1102. Furthermore, with a threshold value which allows acquisition of anedge at the boundary between the regions 1106 and 1107 in the image1102, an edge may be acquired at the boundary between the regions 1108and 1109 in the image 1103. As a result, scanning variations of ascanner and small noise such as Jpeg noise may be detected as edgesdisadvantageously.

FIG. 11 illustrates a configuration for solving those problems. Thevariance value detecting unit 1001 corresponds to a calculation unitconfigured to calculate a variance value of a signal value of an inputimage segmented in character segmentation rectangles. For example, thevariance value may be calculated by the following expression:

${{Variance}\mspace{14mu} V} = \frac{\sum\limits_{i = j}^{n}\left( {{Xi} - {Xave}} \right)^{2}}{n}$

Here, n is a number of pixels of a segmented input image, Xi (i=1, 2, .. . , n) is a signal value of each pixel (L*, a*, b* values in thisexemplary embodiment), and Xave is an average of signal values of thenumber of pixels within the region. A variance value with L*, a*, b*values is acquired in this exemplary embodiment, but an embodiment ofthe present disclosure is not limited thereto. For example, a covariancevalue may be acquired with a*, b* signal values. In the example inputimages 1101, 1102, and 1103 illustrated in FIG. 12, because the inputimage 1101 has a large signal value difference, a greater variance valuemay be acquired. Because the input images 1102 and 1103 have smallersignal value differences, relatively lower variance values may beacquired.

An expression “threshold value for easy edge acquisition” in thefollowing descriptions may refer to a threshold value that allowsdetermination of an edge even when there is a small signal valuedifference between adjacent pixels. On the other hand, an expression“threshold value for difficult edge acquisition” in the followingdescriptions refers to a threshold value with which an edge is notdetermined if a signal value difference is not large and if a signalvalue difference is small.

The edge determination threshold value calculating unit 1002 calculatesa threshold value for performing edge extraction based on the variancevalue calculated by the variance value detecting unit 1001. For example,a threshold value for difficult edge acquisition is allocated to animage having a large variance value as in the image 1101. On the otherhand, a threshold value for easy edge acquisition is allocated for theimages 1102 and 1103.

The edge extracting unit 1003 is a processing unit configured to performan edge extraction process based on the threshold value determined bythe edge determination threshold value calculating unit 1002. Theprocessing may be performed by a general-purpose method by which, forexample, a signal value difference between close pixels is acquired bycomparing them and whether the difference is greater than a specificthreshold value or not is determined. Alternatively, an amount of edgemay be acquired by using a filter for calculating a primary differentialvalue, and whether it is greater than a specific threshold value or notmay be determined.

For segmentation based on a condition calculated by the edgedetermination threshold value calculating unit 1002, a threshold valuefor difficult edge acquisition is allocated to the input image 1101 forthe edge extraction. For example, an example in which the thresholdvalue determined based on a variance value is 5 will be described. Whenthe threshold value is used, an edge between the regions 1104 and 1105may be extracted securely because there is a large signal valuedifference between the regions 1104 and 1105. This may result in an edgeimage 1110. On the other hand, a threshold value for easy edgeacquisition is allocated to the input image 1102 though there is a smallsignal value difference between the regions 1106 and 1107 so that anedge between the regions 1106 and 1107 may be extracted. This may resultin an edge image 1111. A threshold value for easy edge acquisition isallocated to the input image 1103, but the signal value differencebetween the regions 1108 and 1109 is significantly smaller than thesignal value difference between the regions 1106 and 1107. Thus, evenwith a threshold value for easy edge acquisition, an edge presentbetween the regions 1108 and 1109 may not be extracted. This may resultin an edge image 1112.

Next, the edge detecting unit (302) in FIG. 11 will be described withreference to the flowchart in FIG. 13. FIG. 11 will be further referredin the following description.

First, in step 1201, a variance value calculating unit (1001) calculatesa variance value of a signal for an input image (501). In this case,variance values for all three channels may be calculated if the imagehas three channels, or one variance value may be calculated by mergingthem into one channel.

Next, in step 1202, an edge threshold value calculating unit (1002)determines whether the variance values of signals of the imagecalculated in step 1201 are equal to or greater than a predeterminedvalue or not. If they are equal to or greater than the predeterminedthreshold value, a “threshold value for easy edge acquisition” isacquired in step 1203. If they are lower than the predeterminedthreshold value on the other hand, a “threshold value for difficult edgeacquisition” is acquired in step 1204.

Finally, in step 1205, an edge extracting unit (1003) performs an edgeextraction process based on the threshold value determined in step 1203or 1204.

According to this exemplary embodiment, it is configured such that thethreshold value is adaptively changed based on variance values of eachimage segmented for each character segmentation rectangle before an edgeextraction process is performed thereon, as described above. This allowss segmentation with high precision between a “character image difficultto be separated from the background” and a “character image easy to beseparated from the background”.

Fifth Exemplary Embodiment

According to the fourth exemplary embodiment, a method has beendescribed which changes a threshold value based on a variance value of asignal value for threshold value calculation before an edge extractionprocess is performed. In a case where an input image is a polychromeimage with three channels, for example, an equal number of variancevalues to the number of channels may be calculated for use indetermination of a threshold value with high precision. However, in acase where an input image is a gray-scale image and therefore has onechannel, one variance value may be used for the threshold valuecalculation, making it difficult to calculate a threshold value withhigh precision.

According to this exemplary embodiment, the edge detecting unit (302)further includes a black pixel density calculating unit 1004 in additionto the variance value detecting unit 1001, edge determination thresholdvalue calculating unit 1002, and edge extracting unit 1003, asillustrated in FIG. 14. This configuration is also applicable to abinarized image in addition to an input image.

The black pixel density calculating unit 1004 is configured to calculatea ratio of the number of black pixels to a dimension of a charactersegmentation rectangle based on an input binarized image. The number ofblack pixels is counted within an input binarized image, and the countis divided by a dimension of a character segmentation rectangle.

Next, the edge threshold value calculating unit 1002 calculates anoptimum threshold value based on the black pixel density calculated bythe black pixel density calculating unit 1004. Also in this case, athreshold value for edge extraction is calculated based on the blackpixel density similarly to the change of a threshold value for edgeextraction based on a variance value according to the first exemplaryembodiment. More specifically, if the black pixel density is high, a“threshold value for easy edge acquisition” is set. If the black pixeldensity is low, a “threshold value for difficult edge acquisition” isset. Setting in this manner may allow edge extraction with a “thresholdvalue for easy edge acquisition” for precise calculation of edgesbecause a “character over a high-density background” has a high blackpixel density.

It should be noted that both of a threshold value calculated based on avariance value and a threshold value calculated based on a black pixeldensity may be used for calculating a threshold value for edgeextraction though use of either one is also possible. In this case,though a “threshold value for easy edge acquisition” is desirable fromviewpoint of acquisition of more edges, selection of a “threshold valuefor difficult edge acquisition” is also possible. Priority may be givento a threshold value calculated based on a variance value, for example,by switching the weight on the threshold values.

As illustrated in FIG. 15, the edge detecting unit (302) may furtherinclude a number-of-closed-loop calculating unit 1005 in addition to thevariance value detecting unit 1001, edge determination threshold valuecalculating unit 1002, edge extracting unit 1003, and black pixeldensity calculating unit 1004.

The number-of-closed-loop calculating unit 1005 is a calculating unitconfigured to perform a labeling process for calculating the number ofclosed loops formed by successive pixel in a white part of an inputbinarized image.

Next, the edge threshold value calculating unit 1002 calculates anoptimum threshold value based on the number of closed loops calculatedby the number-of-closed-loop calculating unit 1005. Similarly to thefirst exemplary embodiment again, a threshold value to be used for edgeextraction may be calculated based on the number of closed loops. Morespecifically, for a greater number of closed loops, a “threshold valuefor difficult edge acquisition” is used. For a lower number of closedloops, a “threshold value for easy edge acquisition” is used.

This processing allows calculation of an optimum threshold for edgeextraction even for an image, such as a gray scale image, which has alower number of channels and for which a threshold value for edgeextraction may not be calculated based on a variance of a signal value.

Other Embodiments

Embodiments of the present disclosure can also be realized by a computerof a system or apparatus that reads out and executes computer executableinstructions recorded on a storage medium (e.g., a non-transitorycomputer-readable storage medium) to perform the functions of one ormore of the above-described embodiment(s) of the present disclosure, andby a method performed by the computer of the system or apparatus by, forexample, reading out and executing the computer executable instructionsfrom the storage medium to perform the functions of one or more of theabove-described embodiment(s). The computer may comprise one or more ofa central processing unit (CPU), micro processing unit (MPU), or othercircuitry, and may include a network of separate computers or separatecomputer processors. The computer executable instructions may beprovided to the computer, for example, from a network or the storagemedium. The storage medium may include, for example, one or more of ahard disk, a random-access memory (RAM), a read only memory (ROM), astorage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present disclosure has been described with reference toexemplary embodiments, it is to be understood that the disclosure is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of priority from Japanese PatentApplication No. 2013-262763, filed Dec. 19, 2013, Japanese PatentApplication No. 2014-054163, filed Mar. 17, 2014, and Japanese PatentApplication No. 2014-139870, filed Jul. 7, 2014, which are herebyincorporated by reference herein in their entirety.

What is claimed is:
 1. An image processing apparatus comprising: abinarizing unit configured to generate a binary image by executing abinarization process on an input image; a first determining unitconfigured to determine a character region within the binary image; acharacter segmenting unit configured to acquire a region surrounding acharacter contained in the character region as character segmentationrectangle information; a thinning unit configured to acquire a thinnedimage by executing a thinning process on a region within the binaryimage, the region being identified based on the character segmentationrectangle information; an edge detecting unit configured to acquire anedge detected image by executing an edge detection process on the regionidentified based on the character segmentation rectangle information; alogical operation unit configured to take a logical AND of the thinnedimage and the edge detected image; a second determining unit configuredto determine, based on a result of the logical AND performed by thelogical operation unit, whether the character identified based on thecharacter segmentation rectangle information is a character to beseparated from a background by the binarization process or not; and animage processing unit configured to perform a predetermined imageprocess for a character on the region being identified based on thecharacter segmentation rectangle information based on determination ofthe second determining unit that the character identified based on thecharacter segmentation rectangle information is a character to beseparated from a background by the binarization process, wherein theimage processing unit does not perform the predetermined image processfor a character on the region being identified based on the charactersegmentation rectangle information based on determination of the seconddetermining unit that the character identified based on the charactersegmentation rectangle information is not a character to be separatedfrom a background by the binarization process, wherein the units areimplemented by one or more processors, a circuitry, or a combinationthereof.
 2. The image processing apparatus according to claim 1, furthercomprising: a first compression unit configured to execute a binaryreversible compression process on a character determined by the seconddetermining unit as a character to be separated from a background by thebinarization process; and a second compression unit configured toexecute a multi-valued non-reversible compression on a characterdetermined by the second determining unit as a character not to beseparated from the background by the binarization process, wherein thefirst compression unit and the second compression unit are implementedby one or more processors, a circuitry or a combination thereof.
 3. Theimage processing apparatus according to claim 2, further comprising acolor extracting unit configured to extract a character color of thecharacter determined by the second determining unit as a character to beseparated from the background by the binarization process, wherein thecolor extracting unit is implemented by one or more processors, acircuitry or a combination thereof.
 4. The image processing apparatusaccording to claim 1, further comprising a reducing unit configured toacquire a reduced multi-valued image by reducing the input image,wherein the edge detecting unit acquires the edge detected image byexecuting an edge detection process on the region identified based onthe character segmentation rectangle information in the reducedmulti-valued image, wherein the reducing unit is implemented by one ormore processors, a circuitry or a combination thereof.
 5. The imageprocessing apparatus according to claim 1, wherein the edge detectingunit acquires the edge detected image by executing an edge detectionprocess on a region identified based on the character segmentationrectangle information in the input image.
 6. The image processingapparatus according to claim 1, wherein the logical operation unitgenerates a logical AND image by taking a logical AND of the thinnedimage and the edge detected image; and the second determining unitdetermines whether a character identified based on the charactersegmentation rectangle information is a character to be separated from abackground by the binarization process or not based on a number of blackpixels contained in the logical AND image generated by the logicaloperation unit.
 7. The image processing apparatus according to claim 1,further comprising a re-binarizing unit configured to execute are-binarization process on a character determined by the seconddetermining unit as a character not to be separated from a background bythe binarization process.
 8. The image processing apparatus according toclaim 1, wherein the thinning unit changes an amount of reduction of thethinning process in accordance with a size of a region identified basedon the character segmentation rectangle information.
 9. An imageprocessing method executed by at least one processor, the imageprocessing method comprising: generating a binary image by executing abinarization process on an input image; determining a character regionwithin the binary image; acquiring a region surrounding a charactercontained in the character region as character segmentation rectangleinformation; acquiring a thinned image by executing a thinning processon a region within the binary image, the region being identified basedon the character segmentation rectangle information; acquiring an edgedetected image by executing an edge detection process on the regionidentified based on the character segmentation rectangle information;taking a logical AND of the thinned image and the edge detected image;determining, based on a result of the logical AND performed by thetaking, whether the character identified based on the charactersegmentation rectangle information is a character to be separated from abackground by the binarization process or not; and performing apredetermined image process for a character on the region beingidentified based on the character segmentation rectangle informationbased on determination of the determining that the character identifiedbased on the character segmentation rectangle information is a characterto be separated from a background by the binarization process, whereinthe performing does not perform the predetermined image process for acharacter on the region being identified based on the charactersegmentation rectangle information based on determination of thedetermining that the character identified based on the charactersegmentation rectangle information is not a character to be separatedfrom a background by the binarization process.